Imaging system and method

ABSTRACT

A system and method may include capturing a multi-channel polarimetric image and a multi-channel RGB image of a scene by a color polarimetric imaging camera. A multi-channel hyperspectral image may be synthesized from the multi-channel RGB image and concatenated with the multi-channel polarimetric image to create an integrated polarimetric-hyperspectral image. Scene properties within the integrated polarimetric-hyperspectral image may be disentangled.

INTRODUCTION

This disclosure is related to situational awareness and automatedvehicle control.

Imaging systems are known to monitor the region surrounding a vehiclefor improving situational awareness. Such systems may provide scenereconstruction, including object and clear path determinations,providing such determinations for operator alerts and use as controlinputs. Such systems may be useful in vehicle controls ranging from fulloperator control to full autonomous control. Vehicles may includevarious advanced driver-assistance systems (ADAS) which may control someor all aspects of vehicle navigational dynamics in response to inputsfrom such imaging systems.

Imaging systems may provide scene discrimination and objectclassification. Scene depth information may be provided by time offlight measurements, such as by radio detection and ranging (RaDAR) andlight detection and ranging (LiDAR) systems. However, RaDAR and LiDARsystems may have limited spatial resolution when compared to imagingsystems. Thus, combining information from both types of systems wherethere is spatial information mismatch is challenging. Color polarimetricimaging devices are known which provide combined color and polarimetricinformation at equivalent imaging system spatial resolutions. Colorpolarimetric imaging may enable improvements in combined scenediscrimination, object classification and depth. However, such imagingdevices provide scene property information that is substantially tangledand hence difficult to extract.

SUMMARY

In one exemplary embodiment, an apparatus may a color polarimetricimaging camera providing a multi-channel polarimetric image of a scene.The color polarimetric imaging camera may also provide a multi-channelRGB image of the scene. A processor may be adapted to receive themulti-channel polarimetric image and the multi-channel RGB image fromthe color polarimetric imaging camera and may be configured tosynthesize a multi-channel hyperspectral image from the multi-channelRGB image, concatenate the multi-channel polarimetric image and themulti-channel hyperspectral image to create an integratedpolarimetric-hyperspectral image, and disentangle scene propertieswithin the integrated polarimetric-hyperspectral image.

In addition to one or more of the features described herein,disentangling scene properties within the integratedpolarimetric-hyperspectral image may include extracting individual sceneproperties from the integrated polarimetric-hyperspectral image.

In addition to one or more of the features described herein,disentangling scene properties within the integratedpolarimetric-hyperspectral image may include receiving the integratedpolarimetric-hyperspectral image at an input layer of a trained neuralnetwork, and extracting individual scene properties from the integratedpolarimetric-hyperspectral image through the trained neural network.

In addition to one or more of the features described herein,disentangling scene properties within the integratedpolarimetric-hyperspectral image may include receiving the integratedpolarimetric-hyperspectral image at an input layer of a trained neuralnetwork, an generating a depth mapped spatial image through the trainedneural network.

In addition to one or more of the features described herein, theindividual scene properties may include illumination, material andsurface orientation.

In addition to one or more of the features described herein,synthesizing a multi-channel hyperspectral image from the multi-channelRGB image may include a sparse application of an overcomplete dictionarytechnique.

In addition to one or more of the features described herein, themulti-channel polarimetric image, the multi-channel RGB image, and themulti-channel hyperspectral image may include spatial identity.

In another exemplary embodiment, a method may include capturing amulti-channel polarimetric image and a multi-channel RGB image of ascene by a color polarimetric imaging camera, synthesizing amulti-channel hyperspectral image from the multi-channel RGB image by aprocessor, concatenating the multi-channel polarimetric image and themulti-channel hyperspectral image to create an integratedpolarimetric-hyperspectral image by the processor, and disentanglingscene properties within the integrated polarimetric-hyperspectral image.

In addition to one or more of the features described herein,disentangling scene properties within the integratedpolarimetric-hyperspectral image may include extracting individual sceneproperties from the integrated polarimetric-hyperspectral image.

In addition to one or more of the features described herein,disentangling scene properties within the integratedpolarimetric-hyperspectral image may include receiving the integratedpolarimetric-hyperspectral image at an input layer of a trained neuralnetwork, and extracting individual scene properties from the integratedpolarimetric-hyperspectral image through the trained neural network.

In addition to one or more of the features described herein,disentangling scene properties within the integratedpolarimetric-hyperspectral image may include receiving the integratedpolarimetric-hyperspectral image at an input layer of a trained neuralnetwork, an generating a depth mapped spatial image through the trainedneural network.

In addition to one or more of the features described herein, theindividual scene properties may include illumination, material andsurface orientation.

In addition to one or more of the features described herein,synthesizing a multi-channel hyperspectral image from the multi-channelRGB image may include a sparse application of an overcomplete dictionarytechnique.

In addition to one or more of the features described herein, themulti-channel polarimetric image, the multi-channel RGB image, and themulti-channel hyperspectral image may include spatial identity.

In yet another exemplary embodiment, a method may include capturing amulti-channel polarimetric image and a multi-channel RGB image of ascene exterior of a vehicle by a color polarimetric imaging camera onthe vehicle, synthesizing a multi-channel hyperspectral image from themulti-channel RGB image by a processor, concatenating the multi-channelpolarimetric image and the multi-channel hyperspectral image to createan integrated polarimetric-hyperspectral image by the processor,receiving the integrated polarimetric-hyperspectral image at an inputlayer of a trained neural network by the processor, at least one of (i)extracting individual scene properties from the integratedpolarimetric-hyperspectral image through the trained neural network bythe processor, and (ii) generating a depth mapped spatial image throughthe trained neural network by the processor, and performing a vehiclecontrol operation to control the vehicle in response to at least one ofscene properties and depth mapped spatial images.

In addition to one or more of the features described herein, theindividual scene properties may include illumination, material andsurface orientation.

In addition to one or more of the features described herein, the depthmapped spatial images comprise depth channel images and multi-channelRGB images.

In addition to one or more of the features described herein,synthesizing a multi-channel hyperspectral image from the multi-channelRGB image may include a sparse application of an overcomplete dictionarytechnique.

In addition to one or more of the features described herein, themulti-channel polarimetric image, the multi-channel RGB image, and themulti-channel hyperspectral image may include spatial identity.

The above features and advantages, and other features and advantages ofthe disclosure are readily apparent from the following detaileddescription when taken in connection with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

Other features, advantages, and details appear, by way of example only,in the following detailed description, the detailed descriptionreferring to the drawings in which:

FIG. 1 illustrates an exemplary system for sensing the environment inthe vicinity of a vehicle and controlling the vehicle navigationaldynamics, in accordance with the present disclosure;

FIG. 2 shows an arrangement of a 4×4 (16 pixel) calculation unitincluding a polarizer array layer and common color filter array of anexemplary color polarimetric camera sensor, in accordance with thepresent disclosure;

FIG. 3 shows a 2×2 (4 pixel) calculation unit of an exemplary colorpolarimetric camera sensor, in accordance with the present disclosure;

FIG. 4 shows a view of a 4×4 (16 pixel) calculation unit of an exemplarycolor polarimetric camera sensor, in accordance with the presentdisclosure;

FIG. 5 illustrates a graphical representation of the complex data setsaccounting for the image properties influencing hyperspectral andpolarimetric images, in accordance with the present disclosure; and

FIG. 6 illustrates an exemplary process for generating a depth mappedspatial image from a single color polarimetric camera, in accordancewith the present disclosure.

DETAILED DESCRIPTION

The following description is merely exemplary in nature and is notintended to limit the present disclosure, its application or uses.Throughout the drawings, corresponding reference numerals indicate likeor corresponding parts and features. As used herein, control module,module, control, controller, control unit, electronic control unit,processor and similar terms mean any one or various combinations of oneor more of Application Specific Integrated Circuits (ASIC), electroniccircuits, central processing units (preferably microprocessors) andassociated memory and storage (read only memory (ROM), random accessmemory (RAM), electrically programmable read only memory (EPROM), harddrive, etc.), graphic processing units, or microcontrollers executingone or more software or firmware programs or routines, combinationallogic circuits, input/output circuitry and devices (I/O) and appropriatesignal conditioning and buffer circuitry, high speed clock, analog todigital (A/D) and digital to analog (D/A) circuitry and other componentsto provide the described functionality. A control module may include avariety of communication interfaces including point-to-point or discretelines and wired or wireless interfaces to networks including wide andlocal area networks, on vehicle controller area networks and in-plantand service-related networks. Functions of the control module as setforth in this disclosure may be performed in a distributed controlarchitecture among several networked control modules. Software,firmware, programs, instructions, routines, code, algorithms and similarterms mean any controller executable instruction sets includingcalibrations, data structures, and look-up tables. A control module hasa set of control routines executed to provide described functions.Routines are executed, such as by a central processing unit, and areoperable to monitor inputs from sensing devices and other networkedcontrol modules and execute control and diagnostic routines to controloperation of actuators. Routines may be executed at regular intervalsduring ongoing engine and vehicle operation. Alternatively, routines maybe executed in response to occurrence of an event, software calls, or ondemand via user interface inputs or requests.

Turning now to FIG. 1, a block diagram illustrates an exemplary system100 for sensing the environment in the vicinity of a vehicle andcontrolling the vehicle navigational dynamics in response thereto.System 100 may include vehicle 101, a processor 140, a colorpolarimetric camera 120, a memory 145, a vehicle controller 130, athrottle controller 155, a brake controller 160 and a steeringcontroller 170. Processor 140 may be a graphics processing unit. Thevehicle controller 130 may generate control signals for coupling toother vehicle system controllers, such as a throttle controller 155, abrake controller 160 and a steering controller 170 in order to controlthe operation of the vehicle in accordance with ADAS controls. Thevehicle controller 130 may be operative to adjust the speed of thevehicle via the throttle controller 155 or to apply the friction brakesvia the brake controller 160 in response to inputs generated by theprocessor 140, for example depth mapped spatial images. The vehiclecontroller 130 may be operative to adjust the direction of the vehiclecontrolling the vehicle steering via the steering controller 170 inresponse to inputs generated by the processor 140.

The system 100 may utilize the color polarimetric camera 120, amongother sensors, to detect objects in proximity to the vehicle. Sensorfusion may be performed to provide accurate detection, classification,tracking, etc. of external objects as well as calculation of appropriateattributes such as relative velocities, accelerations, and the like. Thecolor polarimetric camera 120 is operative to capture an image within afield of view (FOV) which may include static and dynamic objects withinthat FOV. Image processing techniques may be used to locate and classifyobjects within the FOV. The FOV generally correlates to the drivingscene or portion of the driving scene as limited by the FOV.

The color polarimetric camera 120 may be operative to capture an image,or a series of images of a field of view proximate to the vehicle 101.The series of images may be used to generate a video or the like of thefield of view over time. The color polarimetric camera 120 may beoperative to passively collect polarimetric data and may be equippedwith a polarization array layer over a common color filter array (CFA)(Bayer filter). In an exemplary embodiment, the polarimetric camera 120may be operative to collect 0°, 45°, 90° and 135° polarization angledata for each of four quadrant colors (RGGB) of the common color filterarray. Thus, in this exemplary embodiment, the color polarimetric camera120 may provide 16 different polarization values per calculation unit.In this exemplary embodiment, the processor 140 may be operative toreceive, from the color polarimetric camera 120, the raw polarizationangle data (I₀, I₄₅, I₉₀, and I₁₃₅) from each of the color quadrants(RGGB) of the common color filter array of the color polarimetric camera120. In addition, the processor 140 may receive red, green and blue(RGB) light information from each of the color quadrants (RGGB) of thecommon color filter array of the color polarimetric camera 120. Thus, inthe exemplary embodiment, the electromagnetic radiation from the fieldof view proximate to the vehicle is captured by the polarimetric camera120 as color and polarization data for each pixel and is coupled to theprocessor 140. The digitized raw data output from the polarimetriccamera 120 may include seven information channels where three channelsare RGB color channels and four channels are polarization channels.

Turning now to FIGS. 2, 3 and 4, an exemplary color polarimetric camerasensor 250 may include a polarizer array layer 260 made of nanowire-gridpatterns placed on the focal-plane-array (FPA) above a photodiode layer.The polarizer array layer 260 could be inherently implemented as part ofthe sensor fabrication process. Alternatively, the polarizer array layer260 could be designed and manufactured and then applied to the FPA. FIG.2 illustrates one 4×4 (16 pixel) calculation unit of color polarimetriccamera sensor 250, it being understood that color polarimetric camerasensor 250 includes a plurality of such calculation units in accordancewith the desired resolution of the color polarimetric camera 120.

In order to measure light color intensity (i.e., red, green, blue (RGB)data) as well as polarimetric data, the polarizer array layer 260 may beimplemented on top of a common color filter array 255. Each of the colorquadrants (RGGB) of the CFA 255 may correspond to a 2×2 (4 pixel)calculation unit Q (FIG. 3) wherein each pixel corresponds to arespective one of the four polarization angles 0°, 45°, 90° and 135° ofthe polarizer array layer 260. In an exemplary embodiment, four 2×2 (4pixel) calculation units (Q1, Q2, Q3, Q4), each corresponding to arespective one of the four color quadrants (RGGB) of the CFA, are usedto form a 4×4 (16 pixel) calculation unit (FIG. 4).

FIG. 3 illustrates an exemplary 2×2 (4 pixel) calculation unit Qallowing measurement of three Stokes parameters (denoted S0, S1, S2) andderivation of Angle of Linear Polarization (AoLP) and Degree of LinearPolarization (DoLP). In this example, the polarization data iscalculated independently for each color of the four colors of the CFAand the camera output is a scene image with five channels: R, G, B,AoLP, and DoLP. Thus the raw data received from each of the angledpolarizers, I₀, I₄₅, I₉₀, and I₁₃₅ are used to determine the Stokesparameters S₀, S₁, S₂ according to EQS. 1-3.S ₀ =I ₀ +I ₉₀ =I ₄₅ +I ₁₃₅  [1]S ₁ =I ₀ −I ₉₀  [2]S ₂ =I ₄₅ −I ₁₃₅  [3]

In turn, the stokes parameters may be used to generate a camera outputof a field of view image with 3 channels: Intensity, AoLP, and DoLPaccording to EQS. 4 and 5.

$\begin{matrix}{{DoLP} = {{2\frac{\sqrt{s_{1}^{2} + s_{2}^{2}}}{s_{0}}} \in \left\lbrack {0,1} \right\rbrack}} & \lbrack 4\rbrack \\{{AoLP} = {{{0.5}{\tan^{- 1}\left( \frac{s_{2}}{s_{1}} \right)}} \in \left\lbrack {0{{^\circ}\ldots}\mspace{14mu} 180{^\circ}} \right\rbrack}} & \lbrack 5\rbrack\end{matrix}$

FIG. 5 illustrates, on the left, well-known scene properties ofillumination 501, material 503 and surface orientation 505. On theright, hyperspectral imaging 507 and polarimetric imaging 509 arerepresented. Illumination 501 is known to materially influence bothhyperspectral imaging 507 and polarimetric imaging 509 as shown.Material 503 is also known to materially influence both hyperspectralimaging 507 and polarimetric imaging 509 as shown. However, surfaceorientation is known to materially influence only polarimetric imaging509 but not hyperspectral imaging 507 as shown. Further, the relativeinfluence of each scene property 501-505 is exemplified in the relativeweights of the connecting arrows as illustrated. Thus, it is appreciatedthat hyperspectral imaging 507 is influenced relatively equally by thescene properties of illumination 501 and material 503, whereaspolarimetric imaging 509 is influenced by the scene property of material503 substantially equivalent to its influence upon hyperspectral imaging507, to a relatively lesser degree by the scene property of illumination501 and to a relatively greater degree by the scene property of surfaceorientation 505. FIG. 5 represents a simplistic, graphicalrepresentation of the complex data sets accounting for the sceneproperties influencing hyperspectral and polarimetric images.

FIG. 6, in accordance with one embodiment, illustrates an exemplaryprocess for generating a depth mapped spatial image from a single colorpolarimetric camera 120. The process may be executed by processor 140(FIG. 1). The color polarimetric camera 120 provides a multi-channelpolarimetric image 601 of the scene within the field of view of thecamera. Multi-channel polarimetric image 601 in the present embodimentincludes four channels of polarimetric information, one for each of theraw polarization data (I₀, I₄₅, I₉₀, and I₁₃₅). Each channel of themulti-channel polarimetric image 601 occupies the same spatialdimensions corresponding to the resolution of the color polarimetriccamera 120. Color polarimetric camera 120 also provides a multi-channelRGB image 603 of the scene within the field of view of the camera.Multi-channel RGB image 603 in the present embodiment includes threechannels of color information, one for each of the RGB colors. Eachchannel of the multi-channel RGB image 603 occupies the same spatialdimensions corresponding to the resolution of the color polarimetriccamera 120. In accordance with the present disclosure, a multi-channelhyperspectral image 605 is synthesized from the RGB image 603. Eachchannel of the multi-channel hyperspectral image 605 occupies the samespatial dimensions corresponding to the resolution of the colorpolarimetric camera 120. Each channel of the multi-channel polarimetricimage 601, the multi-channel RGB image 603 and the multi-channelhyperspectral image 605 may be saved into memory 145, and advantageouslymay be stored as arrays of equivalent spatial identity. Thus, it isappreciated that the multi-channel polarimetric image 601, themulti-channel RGB image 603, and the multi-channel hyperspectral image605 share spatial identity.

It is known that each pixel of the RGB image 603 may be sampled at aplurality of spectral points or regions, for example 100, from within apredetermined range of frequencies or wavelengths of interest, forexample, within a visible spectrum from about 400 nm to about 700 nm.From each pixel, therefore, may be extracted a 100-dimension vectorwherein each dimension is a different spectral point or region. Thus,each pixel RGB may be represented by a 100-dimension vector. However, ithas been recognized that while the entire spectrum may requirerepresentation by, for example, a high (e.g. 100) dimensional space, amuch lower dimensional space (e.g. 3-8) is actually required tosatisfactorily represent each pixel. Therefore, in one embodiment, anovercomplete dictionary of 100-dimension vectors may be built and eachactual RGB point may have a sparse linear combination of only several ofthese vectors. The overcomplete dictionary enforces sparseness on therepresentation. From this, an RGB projection—going from 100 dimensionsto, for example, 3-8 dimensions—is a simple deterministic function.Given a new RGB image from the color polarimetric camera 120,coefficients of the dictionary after it was projected to RGB can bedetermined to return a sparse representation in the dictionary of aminimal linear combination of several ones of the entire plurality ofvectors. The hyperspectral image may thus be synthesized with a linearcombination of the vectors at each pixel in the image. The foregoingrepresents but one known technique, referred to as a sparse applicationof an overcomplete dictionary technique, known to those skilled in theart to reconstruct a multi-channel hyperspectral image from an RGBimage.

Subsequent to synthesizing the multi-channel hyperspectral image 605,the spatial identity of the multi-channel polarimetric image 601 and themulti-channel hyperspectral image 605 allows for simple linearconcatenation of all channels of the respective images (601, 605) intoan integrated image 607. The integrated image may next be input todownstream processing 609 to disentangle the underlying illumination,material and surface orientation scene properties. Such processing anddisentanglement may further enhance the ability to independently senseand utilize each of the scene properties. Thus, in addition to fullthree-dimensional perception of a scene, object or feature, material maybe distinguishable. For example, a preceding road patch feature may bedistinguished as black ice vs. paper debris, which distinction may berelevant to operator notifications and/or ADAS functions. In oneembodiment, downstream processing may include a trained deep neuralnetwork to generate a depth mapped spatial image. In one embodiment,processor 140 may employ a convolutional encoder-decoder to transformintegrated image information into depth mapped spatial images. Forexample, the encoder may include several layers, each containingconvolution filters of various sizes, pooling blocks, normalizationblocks, and a non-linear activation function. Each layer may output aset of feature maps, also known as channels. The encoder may receive theintegrated image and generate a low dimension representation. Thedecoder may reverse the encoder's operation and may also includemultiple layers, each containing convolution, pooling, normalization,and a non-linear activation function. There may be connections betweenlayers in the encoder and corresponding layers in the decoder. In oneexemplary embodiment, the encoder-decoder architecture may resemble aU-net convolutional network for image segmentation. The network inputmay include integrated images with a predetermined number of totalchannels including channels of hyperspectral data and channels ofpolarization data. The network output may include a single channelrepresenting spatially accurate depth at the pixel level in a depthchannel image. Alternatively, or additionally, the network output mayinclude one or more additional channels representing spatially accuratemetrics of independent scene properties, for example illumination,material and surface orientation in respective scene property images.Alternatively, or additionally, multiple networks may independentlyoutput respective channels representing spatially accurate metrics ofdepth, illumination, material and surface orientation. Downstreamprocessing 609 may also include reintegration of various image channels,for example concatenation of a depth channel image from the network andthe RGB channels of the raw RGB image to generate a depth mapped spatialimage. Other additional channels, for example one or more of anillumination channel image, a material channel image or a surfaceorientation channel image, may integrate with such depth mapped spatialimages. Such multi-channel spatial images and/or independent channels ofspatially accurate metrics of independent scene properties may provideinput to vehicle controller 130 for use with ADAS controls.

Unless explicitly described as being “direct,” when a relationshipbetween first and second elements is described in the above disclosure,that relationship can be a direct relationship where no otherintervening elements are present between the first and second elements,but can also be an indirect relationship where one or more interveningelements are present (either spatially or functionally) between thefirst and second elements.

It should be understood that one or more steps within a method orprocess may be executed in different order (or concurrently) withoutaltering the principles of the present disclosure. Further, althougheach of the embodiments is described above as having certain features,any one or more of those features described with respect to anyembodiment of the disclosure can be implemented in and/or combined withfeatures of any of the other embodiments, even if that combination isnot explicitly described. In other words, the described embodiments arenot mutually exclusive, and permutations of one or more embodiments withone another remain within the scope of this disclosure.

While the above disclosure has been described with reference toexemplary embodiments, it will be understood by those skilled in the artthat various changes may be made and equivalents may be substituted forelements thereof without departing from its scope. In addition, manymodifications may be made to adapt a particular situation or material tothe teachings of the disclosure without departing from the essentialscope thereof. Therefore, it is intended that the present disclosure notbe limited to the particular embodiments disclosed, but will include allembodiments falling within the scope thereof.

What is claimed is:
 1. An apparatus, comprising: a color polarimetricimaging camera providing a multi-channel polarimetric image of a scene;the color polarimetric imaging camera providing a multi-channel RGBimage of the scene; a processor adapted to receive the multi-channelpolarimetric image and the multi-channel RGB image from the colorpolarimetric imaging camera and configured to: synthesize amulti-channel hyperspectral image from the multi-channel RGB image;concatenate the multi-channel polarimetric image and the multi-channelhyperspectral image to create an integrated polarimetric-hyperspectralimage; and disentangle scene properties within the integratedpolarimetric-hyperspectral image.
 2. The apparatus of claim 1, whereindisentangling scene properties within the integratedpolarimetric-hyperspectral image comprises extracting individual sceneproperties from the integrated polarimetric-hyperspectral image.
 3. Theapparatus of claim 1, wherein disentangling scene properties within theintegrated polarimetric-hyperspectral image comprises: receiving theintegrated polarimetric-hyperspectral image at an input layer of atrained neural network; and extracting individual scene properties fromthe integrated polarimetric-hyperspectral image through the trainedneural network.
 4. The apparatus of claim 3, wherein the individualscene properties comprise illumination, material and surfaceorientation.
 5. The apparatus of claim 1, wherein disentangling sceneproperties within the integrated polarimetric-hyperspectral imagecomprises: receiving the integrated polarimetric-hyperspectral image atan input layer of a trained neural network; and generating a depthmapped spatial image through the trained neural network.
 6. Theapparatus of claim 1, wherein synthesizing a multi-channel hyperspectralimage from the multi-channel RGB image comprises a sparse application ofan overcomplete dictionary technique.
 7. The apparatus of claim 1,wherein the multi-channel polarimetric image, the multi-channel RGBimage, and the multi-channel hyperspectral image comprise spatialidentity.
 8. A method, comprising: capturing a multi-channelpolarimetric image and a multi-channel RGB image of a scene by a colorpolarimetric imaging camera; synthesizing a multi-channel hyperspectralimage from the multi-channel RGB image by a processor; concatenating themulti-channel polarimetric image and the multi-channel hyperspectralimage to create an integrated polarimetric-hyperspectral image by theprocessor; and disentangling scene properties within the integratedpolarimetric-hyperspectral image.
 9. The method of claim 8, whereindisentangling scene properties within the integratedpolarimetric-hyperspectral image comprises extracting individual sceneproperties from the integrated polarimetric-hyperspectral image.
 10. Themethod of claim 8, wherein disentangling scene properties within theintegrated polarimetric-hyperspectral image comprises: receiving theintegrated polarimetric-hyperspectral image at an input layer of atrained neural network; and extracting individual scene properties fromthe integrated polarimetric-hyperspectral image through the trainedneural network.
 11. The method of claim 10, wherein the individual sceneproperties comprise illumination, material and surface orientation. 12.The method of claim 8, wherein disentangling scene properties within theintegrated polarimetric-hyperspectral image comprises: receiving theintegrated polarimetric-hyperspectral image at an input layer of atrained neural network; and generating a depth mapped spatial imagethrough the trained neural network.
 13. The method of claim 8, whereinsynthesizing a multi-channel hyperspectral image from the multi-channelRGB image comprises a sparse application of an overcomplete dictionarytechnique.
 14. The method of claim 8, wherein the multi-channelpolarimetric image, the multi-channel RGB image, and the multi-channelhyperspectral image comprise spatial identity.
 15. A method, comprising:capturing a multi-channel polarimetric image and a multi-channel RGBimage of a scene exterior of a vehicle by a color polarimetric imagingcamera on the vehicle; synthesizing a multi-channel hyperspectral imagefrom the multi-channel RGB image by a processor; concatenating themulti-channel polarimetric image and the multi-channel hyperspectralimage to create an integrated polarimetric-hyperspectral image by theprocessor; receiving the integrated polarimetric-hyperspectral image atan input layer of a trained neural network by the processor; at leastone of (i) extracting individual scene properties from the integratedpolarimetric-hyperspectral image through the trained neural network bythe processor, and (ii) generating depth mapped spatial images throughthe trained neural network by the processor; and performing a vehiclecontrol operation to control the vehicle in response to at least one ofscene properties and depth mapped spatial images.
 16. The method ofclaim 15, wherein the individual scene properties comprise illumination,material and surface orientation.
 17. The method of claim 15, whereinthe depth mapped spatial images comprise depth channel images andmulti-channel RGB images.
 18. The method of claim 15, whereinsynthesizing a multi-channel hyperspectral image from the multi-channelRGB image comprises a sparse application of an overcomplete dictionarytechnique.
 19. The method of claim 15, wherein the multi-channelpolarimetric image, the multi-channel RGB image, and the multi-channelhyperspectral image comprise spatial identity.